Attorney Docket No. 5034-0001 
Customer Number 28777 

LISTING OF THE CLAIMS 

This listing of claims replaces all prior versions and listings of claims in the application: 
Listing of Claims 

1. (Previously Presented) A coherent optical detection system receiving an 
incoming optical signal in an optical communications network, said system comprising; 
a local oscillator emitting light; 

a phase diverse hybrid for generating two replicas of the incoming signal and two 
replicas of the local oscillator light, said phase diverse hybrid combining the first replica 
of the incoming optical signal and the first replica of the local oscillator light into a first 
output and combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output and wherein said local oscillator 
does not have to be phase locked to the incoming optical signal; 

wherein the phase relationship between the optical signal and the local oscillator 
light in the first output is different from 0 degrees and different from 180 degrees 
compared to the phase relationship between the local oscillator light and the optical 
signal in the second output and the state of polarization of the optical signal relative to 
the local oscillator light in the first output is not orthogonal to the state of polarization of 
the optical signal relative to the local oscillator light in the second output; and 

two photodetectors communicating with the phase diverse hybrid, wherein said 
two photodetectors receive optical signals from the two outputs and convert them to 
electrical signals; 
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whereby the electrical signals are processed to provide a complex representation 
of the envelope of the electric field of the incoming optical signal or a component of the 
complex representation of the envelope of the electric field of the incoming optical 
signal. 

2. (Original) The coherent optical detection system of claim 1 wherein: 

the electrical signals received by the two photodetectors are digitized by two A/D 
converters; and 

a digital signal processor performs a computation on digital values from the A/D 
converters to provide a complex representation or component thereof of the incoming 
optical signal. 

3. (Original) The coherent optical detection system of claim 1 wherein: 

the electrical signals from the two photodetectors are separately multiplied by two 
electrical waveforms, each waveform being periodic and having the same frequency as 
the difference frequency between the local oscillator light and the incoming optical 
signal; 

the wavefomris having different phases from one another; and 

the electrical signals corresponding to the results of two multiplication operations 

are summed to give an estimate of a component of the complex representation of the 

incoming optical signal. 
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4. (Original) The coherent optical detection system of claim 2 wherein the digital 
signal processor produces an output which is the result of a signal processing operation 
on a plurality of samples over time of the complex envelope of the electric field of the 
incoming optical signal. 

5. (Original) The coherent optical detection system of claim 4 wherein the digital 
signal processor compensates for the chromatic dispersion experienced by the 
incoming optical signal. 

6. (Original) A coherent optical detection system receiving an incoming optical 
signal in a fiber optics network, said system comprising: 

a local oscillator emitting light; 

an optical mixing hybrid for generating at least four replicas of the incoming 
optical signal and four replicas of the local oscillator light, said optical mixing hybrid 
combining the replicas of the incoming optical signal and the replicas of the local 
oscillator light Into at least four outputs and wherein said local oscillator does not have 
to be phase locked to the incoming optical signal; 

wherein four outputs of the at least four outputs of the optical mixing hybrid can 
be selected such that the Jones vector of the optical signal relative to the local oscillator 
light at each of the four selected outputs is distinct from the Jones vector of the optical 
signal relative to the local oscillator light at the other three of the selected outputs; 

four photodetectors communicating with the optical mixing hybrid, wherein said 
four photodetectors receive optical signals from the four outputs; 

Amendment - PAGE 34 of 54 

EUS/J/P/06-9275 



Attorney Docket No. 5034-0001 
Customer Number 28777 

four A/D converters to receive electrical signals from the four photodetectors, 
said four A/D converters digitizing the received electrical signals; and 

a digital signal processor for performing computations on digital values from the 
four A/D converters to obtain Information carried by the Incoming optical signal without 
limitation to the state of polarization of the Incoming optical signal. 

7. (Previously Presented) The coherent optical detection system of claim 6 
wherein: 

the state of polarization of the optical signal relative the local oscillator light at the 
first and second of the selected outputs is not orthogonal, and the state of polarization 
of the optical signal relative the local oscillator light at the third and fourth of the selected 
outputs is not orthogonal, and the state of polarization at the first and second outputs Is 
close to orthogonal to the state of polarization at the third and fourth outputs; and 

the phase relationship between the optical signal and the local oscillator light at 
the first output Is approximately 90 degrees compared to the phase relationship 
between the optical signal and the local oscillator light at the second output, and the 
phase relationship between the optical signal and the local oscillator light at the third 
output is approximately 90 degrees compared to the phase relationship between the 
optical signal and the local oscillator light at the fourth output. 

8. (Original) The coherent optical detection system of claim 6 wherein the digital 
signal processor obtains a Jones vector representation or equivalent representation that 
Includes the amplitude, phase and polarization of the incoming optical signal. 

Amendment - PAGE 35 of 54 

EUS/J/P/06-9275 



Attorney Docket No. 5034-0001 
Customer Number 28777 

9. (Original) The colierent optical detection system of claim 8 wherein: 

the incoming optical signal includes two optical channels being combined so that 
the first optical channel of the two optical channels has a state of polarization close to 
orthogonal to the second optical channel of the two optical channels; and 

the digital signal processor performs computations to obtain information carried 
by each of the two polarization multiplexed channels in the incoming optical signal 
independent of the other channel. 

10. (Original) The coherent optical detection system of claim 6 wherein the 
digital signal processor produces an output which is the result of a signal processing 
operation on a plurality of samples over time of the Jones vector of the incoming optical 
signal. 

11. (Original) The coherent optical detection system of claim 10 wherein 
the digital signal processor compensates for the chromatic dispersion experienced by 
the Incoming optical signal. 

12. (Previously Presented) A coherent optical detection system receiving 
an incoming optical signal in an optical communications network, said system 
comprising: 

two local oscillators, each local oscillator emitting light; 

an optical mixing hybrid for generating at least four replicas of the incoming 
optical signal and two replicas of light from each of the local oscillators, said optical 
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mixing hybrid combining the incoming optical signal and the local oscillator light into at 
least four outputs and wherein said local oscillators do not have to be phase locked to 
the incoming optical signal; 

a first output of the four outputs having a first replica of the optical signal and a 
first replica of light from the first local oscillator of the two local oscillators, wherein the 
state of polarization of the optical signal and the light emitted from the first local 
oscillator have a defined relationship; 

a second output of the four outputs having a second replica of the optical signal 
and a second replica of light from the fn^ local oscillator of the two local oscillators, 
wherein the phase relationship between the optical signal and the local oscillator light in 
the first output is different from 0 degrees and different from 180 degrees compared to 
the phase relationship between the local oscillator light and the optical signal in the 
second output; 

a third output of the four outputs having a third replica of the optical signal and a 
first replica of light from the second local oscillator of the two local oscillators, wherein 
the state of polarization of the light from the second local oscillator with respect to the 
optical signal in the third output is close to orthogonal compared to the state of 
polarization of light from the first local oscillator with respect to the optical signal in the 
first output; 

a fourth output of the four outputs having a fourth replica of the optical signal and 
a second replica of light from the second local oscillator of the two local oscillators, 
wherein the phase relationship between the optical signal and the local oscillator light in 
the third output is different from 0 degrees and different from 180 degrees compared to 
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the phase relationship between the local oscillator light and the optical signal in the 
fourth output; 

at least four photodetectors communicating with the optical mixing hybrid, 
wherein said four photodetectors receive optical signals from the four outputs; 

at least four A/D converters to receive electrical signals from the four 
photodetectors, said four A/D converters digitizing the electrical signals; and 

a digital signal processor for performing computations on digital values from the 
four A/D converters to obtain information carried by the incoming optical signal. 

13. (Previously Presented) A coherent optical detection system receiving 
an incoming optical signal containing infonnation in a fiber optics network, said system 
comprising: 

a local oscillator emitting light; 

an optical mixing hybrid for combining the incoming optical signal and the local 
oscillator light into at least one output; 

a photodetector communicating with the optical mixing hybrid, wherein said 
photodetector receives an optical signal from the output and converts it to an electrical 
signal; 

an A/D converter to receive the electrical signal from said photodetector, said 
A/D converter digitizing the electrical signal; and 

a digital signal processor for perfomning computations on digital values from the 
A/D converter, the digital signal processor producing an output which is the result of a 
signal processing operation on a plurality of samples over time of the complex envelope 
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of the electric field of the incoming optical signal, the output having the infomnation 
contained in the incoming optical signal. 

14. (Original) The coherent optical detection system of claim 13 wherein 
the digital signal processor at least partially reverses the effect of propagation of the 
incoming optical signal through an optical fiber transmission system. 

15. (Original) The coherent optical detection system of claim 14 wherein 
the digital signal processor compensates for the chromatic dispersion of the optical fiber 
transmission system. 

16. (Original) The coherent optical detection system of claim 15 wherein 
the signal processing operation performed by the digital signal processor compensates 
for the chromatic dispersion experienced by the optical signal by applying to the 
complex envelope of the incoming optical signal a convolution with a specified 
mathematical function, the mathematical function being close to the impulse response 
of the transfer function corresponding to a chromatic dispersion equal in magnitude and 
opposite to the chromatic dispersion of the optical fiber transmission system. 

17. (Original) The coherent optical detection system of claim 14 wherein 
the signal processing operation perfomned by the digital signal processor at least 
partially reverses the effect of self phase modulation imposed on the incoming optical 
signal. 
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18. (Original) The coherent optical detection system of claim 13 wherein 
the signal processing operation performed by the digital signal processor at least 
partially reverses the effect of multipath interference imposed on the incoming optical 
signal. 

19. (Original) The coherent optical detection system of claim 13 wherein 
the signal processing operation performed by the digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 

20. (Original) The coherent optical detection system of claim 13 wherein 
the signal processing operation perfomned by the digital signal processor improves the 
quality of the incoming optical signal, the digital signal processor applying an algorithm 
which utilizes parameters that are adjusted to give different signal processing functions, 
and the values of those parameters are chosen for improving the quality of the 
recovered signal. 

21 . (Original) The coherent optical detection system of claim 20 wherein 
the signal processing operation that improves the quality of the recovered signal is a 
feedforward equalization-decision feedback equalization function. 
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22. (Original) The coherent detection system of claim 20 wherein the 
signal processing operation that improves the quality of the recovered signal is a 
maximum likelihood sequence estimation function. 

23. (Original) The coherent optical detection system of claim 13 wherein 
the digital signal processor produces an output which is the result of a signal processing 
operation on a plurality of samples over time of the Jones vector of the incoming optical 
signal. 

24. (Original) The coherent optical detection system of claim 23 wherein 
the digital signal processor compensates for the chromatic dispersion experienced by 
the Incoming optical signal. 

25. (Original) The coherent optical detection system of claim 23 wherein 
the signal processing operation performed by the digital signal processor at least 
partially reverses the effect of polarization mode dispersion imposed on the incoming 
optical signal. 

26. (Original) A coherent optical detection system receiving an incoming 
optical signal having a plurality of wave division multiplexed (WDM) channels, said 
system comprising: 

at least one local oscillator emitting light; 
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an optical mixing liybrid for combining the incoming optical signal and the local 
oscillator light into at least one output; 

a photodetector communicating with the optical mixing hybrid, wherein said 
photodetector receives optical signals from the output; 

an A/D converter to receive electrical signals from said photodetector, said A/D 
converter digitizing the electrical signals; and 

a digital signal processor for perfomiing computations on digital values from the 
A/D converter, the digital signal processor estimating infonnation carried on a first of the 
plurality of WDM channels that takes into account the other WDM channels and 
subtracts crosstalk imposed on the first WDM channel by at least one of the remainder 
of the plurality of WDM channels. 

27. (Original) The coherent optical detection system of claim 26 wherein 
the crosstalk arises because the optical spectrum of the first WDM channels is partially 
overlapped over the optical spectrum of a second WDM channel of the incoming optical 
signal 

28. (Original) The coherent optical detection system of claim 26 wherein 
the crosstalk is cross phase modulation imposed on the first WDM channel by a second 
WDM channel during passage through an optical fiber transmission system. 

29. (Original) The coherent optical detection system of claim 26 wherein 
the crosstalk is caused by four wave mixing occurring when the plurality of WDM 
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channels generates a four wave mixing product wliich at least partially overlaps the 
optical spectrum of the WDM channel that experiences the crosstalk. 

30, (Previously Presented) A method of receiving an incoming optical 
signal in a coherent optical detection system, said method comprising the steps of: 

emitting light from a local oscillator, said local oscillator not requiring a phase lock 
with the incoming optical signal; 

generating two replicas of the incoming signal and two replicas of the local 
oscillator light by a phase diverse hybrid; 

combining, by the phase diverse hybrid, the first replica of the incoming optical 
signal and the first replica of the local oscillator light into a first output; 

combining the second replica of the incoming optical signal and the second 
replica of the local oscillator light into a second output, wherein the phase relationship 
between the optical signal and the local oscillator light in the first output is different from 
0 degrees and different from 180 degrees compared to the phase relationship between 
the local oscillator light and the optical signal in the second output and the state of 
polarization of the optical signal relative to the local oscillator light in the first output is 
not orthogonal to the state of polarization of the optical signal relative to the local 
oscillator light in the second output; 

receiving optical signals from the two outputs by two photodetectors in 
communication with the phase diverse hybrid; and 

converting the optical signals from the two outputs into electrical signals, the 
electrical signals being processed to provide a complex representation of the envelope 
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of the electric field of the incoming optical signal or a component of the complex 
representation of the envelope of the electric field of the incoming optical signal. 

31. (Original) The method of receiving an incoming optical signal of claim 
30 wherein the step of converting the optical signals includes: 

digitizing the electrical signals by two A/D converters; and 

performing a computation on digital values from the A/D converters by a digital 
signal processor to provide a complex representation or component thereof of the 
incoming optical signal. 

32. (Previously Presented) The method of receiving an incoming optical 
signal of claim 30 wherein the step of converting the optical signals includes the steps 
of: 

multiplying the electric signals separately by two electrical waveforms, each 
waveform being periodic and having the same frequency as the difference frequency 
between the local oscillator light and the incoming optical signal and wherein the two 
electrical waveforms have different phases from one another; and 

summing the electrical signals corresponding to the results of the two 
multiplication operations to give an estimate of a component of the complex 
representation of the incoming optical signal. 

33. (Original) A method of receiving an incoming optical signal in a 
coherent optical detection system, said method comprising the steps of: 
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emitting light from a local oscillator, said local oscillator not requiring a phase lock 
with the incoming optical signal; 

generating at least four replicas of the incoming optical signal and four replicas of 
the local oscillator light by an optical mixing hybrid; 

combining, by the optical mixing hybrid, the replicas of the incoming optical signal 
and the replicas of the local oscillator light into at least four outputs, wherein four 
outputs of the at least four outputs of the optical mixing hybrid can be selected such that 
the Jones vector of the optical signal relative to the local oscillator light at each of the 
four selected outputs is distinct from the Jones vector of the optical signal relative to the 
local oscillator light at the other three of the selected outputs; 

receiving optical signals from the four outputs by four photodetectors in 
communication with the optical mixing hybrid; 

digitizing electrical signals, by four A/D converters, from the four photodetectors; 

and 

performing computations, by a digital signal processor, on digital values from the 
four A/D converters to obtain infomiation carried by the incoming optical signal without 
limitation to the state of polarization of the incoming optical signal. 

34. (Previously Presented) The method of receiving an incoming optical 
signal of claim 33 wherein: 

the state of polarization of the optical signal relative the local oscillator light at the 
first and second of the selected outputs is not orthogonal, and the state of polarization 
of the optical signal relative the local oscillator light at the third and fourth of the selected 
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outputs is not orthogonal, and the state of polarization at the first and second outputs is 
close to orthogonal to the state of polarization at the third and fourth outputs; and 

the phase relationship between the optical signal and the local oscillator light at 
the first output is approximately 90 degrees compared to the phase relationship 
between the optical signal and the local oscillator light at the second output, and the 
phase relationship between the optical signal and the local oscillator light at the third 
output is approximately 90 degrees compared to the phase relationship between the 
optical signal and the local oscillator light at the fourth output. 

35. (Original) The method of receiving an incoming optical signal of claim 
33 wherein the step of performing computations by the digital signal processor includes 
obtaining a Jones vector representation or equivalent representation that includes the 
amplitude, phase and polarization of the incoming optical signal. 

36. (Original) The method of receiving an incoming optical signal of claim 
33 wherein the step of performing computations by the digital signal processor includes 
producing an output which is the result of a signal processing operation on a plurality of 
samples over time of the Jones vector of the incoming optical signal. 

37. (Previously Presented) A method of receiving an incoming optical 
signal in a coherent optical detection system, said method comprising the steps of: 

emitting light from first and second local oscillators, each local oscillator not 
requiring a phase lock with the incoming optical signal; 
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generating four replicas of the incoming signal and two replicas from each of the 
local oscillators light by an optical mixing hybrid; 

combining, by the optical mixing hybrid into at least four outputs; 

wherein a first output of the four outputs has a first replica of the optical signal 
and a first replica of light from the first local oscillator and the state of polarization of the 
Incoming optical signal and the light emitted from the first local oscillator in the first input 
has a defined relationship; 

wherein a second output of the four outputs includes a second replica of the 
optical signal and a first replica of light from the second local oscillator and the state of 
polarization of the light from the second local oscillator with respect to the optical signal 
in the second input Is close to orthogonal compared to the state of polarization of the 
light from the first local oscillator with respect to the optical signal in the first output; 

receiving optical signals from the two outputs by at least two photodetectors in 
communication with the optical mixing hybrid; 

digitizing electrical signals, by at least two A/D converters, from the at least two 
photodetectors; and 

performing computations, by a digital signal processor, on digital values from the 
at least two A/D converters to obtain information carried by the incoming optical signal. 

38. (Previously Presented) A method of receiving an incoming optical 
signal containing information in a coherent optical detection system, said method 
comprising the steps of: 

emitting light from a local oscillator; 
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combining, by an optical mixing hybrid, the incoming optical signal and the local 
oscillator light into at least one output; 

receiving optical signals from the output by a photodetector in communication 
with the optical mixing hybrid; and 

converting the optical signal into an electrical signal by the photodetector; 

digitizing the electrical signal by an A/D converter; and 

performing computations, by a digital signal processor, on digital values from the 
A/D converter, the digital signal processor producing an output which is the result of a 
signal processing operation on a plurality of samples over time of the complex envelope 
of the electric field of the incoming optical signal, wherein the output has the information 
contained in the incoming optical signal. 

39. (Original) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of propagation of the signal through an optical fiber 
transmission system. 

40. (Original) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
reversing at least partially the effect of multipath interference Imposed on the incoming 
optical signal. 

41 . (Original) The method of receiving an incoming optical signal of claim 
38 wherein the step of performing computations by a digital signal processor includes 
performing an optical filtering function on the complex envelope of the electric field. 
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42. (Previously Presented) A method of receiving an incoming optical 
signal in a coherent optical detection system, said incoming optical signal having a 
plurality of WDM channels, said method comprising the steps of: 

emitting light from at least one local oscillator; 

combining, by an optical mixing hybrid, the incoming optical signal and the local 
oscillator light into at least one output; 

receiving optical signals from the output by a photodetector in communication 
with the optical mixing hybrid; 

converting the optical signal into an electrical signal by the photodetector; 

digitizing the electrical signal from the photodetector by an A/D converters; and 

performing computations, by a digital signal processor, on digital values from the 
A/D converter, the digital signal processor estimating information carried on a first of the 
plurality of WDM channels that takes into account the other WDM channels and 
subtracts crosstalk imposed on the first WDM channel by at least one of the remainder 
of the plurality of WDM channels. 

43. (Original) The method of receiving an incoming optical signal of claim 
42 wherein the crosstalk is cross phase modulation imposed on the first WDM channel 
by a second WDM channel during passage through an optical fiber transmission 
system. 

44. (Original) The method of receiving an incoming optical signal of claim 
42 wherein the crosstalk is caused by four wave mixing occurring when the plurality of 
WDM channels generates a four wave mixing product which at least partially overlaps 
the optical spectrum of the WDM channel that experiences the crosstalk. 
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45. (Previously Presented) A heterodyne coherent optical detection 
system receiving an Incoming optical signal in a fiber optics network, said system 
comprising: 

a local oscillator emitting light; 

a polarization diversity hybrid for generating tw/o replicas of the incoming optical 
signal and two replicas of the light from the local oscillator, said polarization diversity 
hybrid combining the Incoming optical signal and the local oscillator light into at least 
two outputs, wherein the state of polarization of the incoming optical signal relative to 
the light from the local oscillator at the first output is close to orthogonal to the state of 
polarization of the incoming optical signal relative to the light from the local oscillator at 
the second output; 

wherein the Incoming optical signal Includes two optical channels being 
combined so that the first optical channel of the two optical channels has a state of 
polarization close to orthogonal to the second optical channel of the two optical 
channels; 

two photodetectors communicating with the optical mixing hybrid, wherein said 
photodetectors receive optical signals from the outputs and convert them to electrical 
signals; 

two A/D converters to receive the electrical signals from said photodetectors, 
said A/D converters digitizing the electrical signals; and 

a digital signal processor for performing computations on digital values from the 
A/D converters to obtain information carried by each of the two polarization multiplexed 
channels in the incoming optical signal independent of the; other channel. 

46. (Canceled) 
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